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Abstract — Adaptive  estimation  of  the  linear  coefficient  vec¬ 
tor  in  truncated  expansions  is  considered  for  the  purpose  of 
modeling  noisy,  recurrent  signals.  The  block  LMS  (BLMS) 
algorithm,  being  the  solution  of  the  steepest  descent  strategy 
for  minimizing  the  mean  square  error  in  a  complete  signal 
occurrence,  is  shown  to  be  steady-state  unbiased  and  with  a 
lower  variance  than  the  LMS  algorithm.  It  is  demonstrated 
that  BLMS  is  equivalent  to  an  exponential  averager  in  the 
subspace  spanned  by  the  truncated  set  of  basis  functions.  The 
performance  of  the  BLMS  algorithm  is  studied  on  an  ECG 
signal  and  the  results  show  that  its  performance  is  superior 
to  that  of  the  LMS  algorithm. 

Index  Terms — Adaptive  filters,  deterministic  input,  orthog¬ 
onal  expansions,  event-related  signal. 


I.  Introduction 

The  problem  of  noise  reduction  in  recurrent  signals  is 
well-studied  and  has  traditionally  been  solved  by  ensem¬ 
ble  averaging,  or  by  one  of  the  many  variations  on  this 
technique.  The  time  reference  of  each  occurrence  is  of¬ 
ten  synchronized  to  a  known,  external  stimulus;  in  certain 
signals,  however,  the  time  reference  is  difficult  to  observe 
and  therefore  a  fiducial  point  needs  to  be  established  for 
each  occurrence  by  some  kind  of  estimation  procedure.  A 
major  disadvantage  with  ensemble  averaging  is  that  effi¬ 
cient  noise  reduction  is  typically  achieved  at  the  expense 
of  using  a  large  number  of  occurrences  for  averaging.  In 
order  to  better  track  short-term  changes  in  morphology  of 
the  recurrent  signal,  while  still  achieving  a  reduction  of 
the  noise  level,  it  is  desirable  to  develop  methods  which 
incorporate  a  priori  information  on  possible  morphologies. 
More  recently,  modeling  of  each  occurrence  as  a  signal 
which  is  well-described  by  a  truncated  linear  expansion  of 
orthonormal  basis  functions  has  been  studied. 

The  coefficients  of  the  linear  expansion  can  be  estimated 
using  different  approaches.  In  many  situations,  the  mean 
square  error  (MSE)  between  the  observed  signal  and  the 
signal  model  represents  a  suitable  cost  function  since  it  is 
related  to  signal  energy.  The  optimal  coefficients  are  deter¬ 
mined  on  an  individual  occurrence  basis,  thus  constituting 
a  memory-less  estimation.  By  introducing  memory  in  the 
estimator,  the  variance  of  the  coefficient  estimates  can  be 
considerably  reduced  while  the  capability  of  tracking  sig¬ 
nal  changes  in  a  noisy  environment  is  still  available.  Sev¬ 
eral  papers  have  been  presented  in  the  area  of  biomedical 
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signal  processing  where  an  adaptive  solution  based  on  the 
LMS  algorithm  is  suggested,  see  e.g.  [1],  The  reference 
inputs  to  the  LMS  algorithm  are  deterministic  functions 
and  defined  by  a  periodically  extended,  truncated  set  of  or¬ 
thonormal  basis  functions. 

In  these  papers,  the  LMS  algorithm  operates  on  an  “in¬ 
stantaneous”  basis  such  that  the  weight  vector  is  updated 
for  every  new  sample  based  on  an  instantaneous  gradient 
estimation.  In  a  recent  study,  however,  a  steady-state  con¬ 
vergence  analysis  for  the  LMS  algorithm  with  determin¬ 
istic  reference  inputs  showed  that  the  steady-state  weight 
vector  is  biased,  and  thus  the  adaptive  estimate  does  not  ap¬ 
proach  the  Wiener  solution  [2],  To  handle  this  drawback, 
we  consider  another  strategy  for  estimating  the  coefficients 
of  the  linear  expansion,  namely  the  block  LMS  (BLMS) 
algorithm  in  which  the  coefficient  vector  is  updated  only 
once  every  occurrence  based  on  a  block  gradient  estima¬ 
tion.  The  BLMS  algorithm  has  already  been  proposed  for 
the  case  with  random  reference  inputs  and  has,  when  the 
input  is  stationary,  the  same  steady-state  misadjustment 
and  convergence  speed  as  the  LMS  algorithm  [3,4],  A 
major  advantage  of  the  block,  or  the  transform  domain, 
LMS  algorithm  is  that  the  input  signals  are  approximately 
uncorrelated  (or  orthogonal  in  a  more  general  sense).  To 
the  best  of  our  knowledge,  block  adaptation  has  not  been 
considered  previously  within  the  context  of  deterministic 
reference  input  signals. 

The  selection  of  orthonormal  basis  functions  is. 
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basis  functions 

yt=Twt  (l) 

where  T  is  a  N  x  p  matrix  whose  columns  are  the  basis 

functions  and  w/.  is  the  px  1  coefficient  vector  with  p  <  N. 
One  approach  to  find  the  linear  optimal  coefficient  vector 
wj!  is  to  minimize  the  cost  function  defined  by  the  mean 
square  error  between  d/,.  and  y/,. 

Jk=E  |(dfc -Twfc)T(dfc -Twfc)|  .  (2) 

Applying  differentiation  we  obtain  that  the  Wiener  solu¬ 
tion  for  the  linear  coefficient  vector  is 

w°k=TTE{dk}.  (3) 

This  solution  can  be  easily  understood  because  the  optimal 
signal  description  in  the  transform  domain  is  the  projection 
of  the  expected  value  of  the  observed  signal. 

The  observed  signal  dk  is  commonly  contaminated  by 
noise.  Assuming  an  additive-noise  model,  each  signal  oc¬ 
currence  d/.  can  be  decomposed  as  dk  =  sk  +  nowhere 
sk  is  a  deterministic  signal  and  ii;.  is  zero-mean  random 
noise.  Since  the  clean  signal  sk  is  unavailable,  w‘k  needs  to 
be  estimated  according  to  (3)  from  the  observed  signal  d/.. 
A  very  simple  way  is  to  approximate  E  {d/,. }  ~  dk  in  (3), 
implying  that  the  linear  coefficient  vector  is  estimated  by 

wip=TTdfc,  (4) 

where  IP  denotes  the  inner  product  between  each  basis 
function  and  the  observed  signal.  This  kind  of  estimation 
is  memory-less  since  only  information  from  the  k-th  occur¬ 
rence  is  used  to  estimate  E  {d/,.},  and  as  a  result,  sudden 
changes  in  signal  shape  can  be  tracked.  On  the  other  hand, 
wj1’  will  be  sensitive  to  the  presence  of  noise. 

III.  The  BLMS  Estimation 

One  way  to  reduce  the  influence  of  noise  is  to  include 
adaptive  algorithms  in  the  coefficient  estimation  since 
this  type  of  estimators  has  memory  of  previous  occur¬ 
rences.  When  the  deterministic  signal  is  repetitive  with 
slow  occurrence-to-occurrence  shape  changes,  the  amount 
of  noise  can  be  reduced  at  the  expense  of  a  slower  conver¬ 
gence.  The  trade-off  between  convergence  speed  and  sig¬ 
nal  to  noise  ratio  improvement  is  controlled  by  the  memory 
used  in  the  estimation. 

The  structure  of  the  vector-based  adaptive  filter  is  shown 
in  Fig.  1.  The  primary  input,  dk  =  +  n^,  consists  of 

successive  concatenated  signal  occurrences,  not  necessar¬ 
ily  obtained  in  a  contiguous  fashion.  For  the  steady-state 
analysis  of  the  algorithm  we  assume  that  the  deterministic 
signal  S/,  remains  unchanged  during  all  occurrences,  i.e., 
sk  =  s.  In  practice,  sk  will  be  occurrence-variant,  and  the 
algorithm  will  track  signal  changes  in  a  finite  adaptation 
time.  The  adaptive  system  estimates  at  each  signal  occur¬ 
rence  the  amount  of  each  reference  input  (columns  of  the 
N  xp  matrix  T)  present  in  the  input  signal  dk. 


Bfc  —  sfc  T  nfe 


— 


yt=Twt  i+  e 


w  k 


Adaptive 

algorithm 


Fig.  1 .  Adaptive  block- wise  estimation  of  truncated  expansions. 


In  order  to  minimize  the  cost  function  in  (2),  the  optimal 
weight  vector  can  be  estimated  by  using  an  iterative  algo¬ 
rithm  based  on  the  steepest  descent  strategy:  the  weight 
vector  is  updated  once  every  occurrence  according  to 

dJk 

Wfc+i  =  w k-  n  - - ,  (5) 

dwk 

where  p  is  the  step-size  that  controls  stability  and  conver¬ 
gence  speed  of  the  algorithm.  The  weight  vector  update 
equation  can  be  obtained  using  the  classical  gradient  ap¬ 
proximation  as 


wfc+1  =  (1  -  2p)  wk  +  2pTTdk  .  (6) 

This  algorithm  is  named  BLMS  because  it  is  equivalent  to 
the  LMS  algorithm  but  with  a  block-wise  gradient  estima¬ 
tion.  In  other  words,  the  BLMS  is  equivalent  to  exponen¬ 
tial  averaging  in  the  subspace  spanned  by  T.  Note  that  the 
BLMS  is  equivalent  to  IP  when  p  =  \. 

We  will  now  consider  the  bias  and  variance  of  the  BLMS 
algorithm,  since  these  quantities  are  useful  for  comparison 
with  other  estimation  methods.  The  weight  error  vector  at 
the  fc-th  occurrence  \k  =  w/.  —  w°  can  be  written  as 

fc-i 

v*  =  (1  -  2p)k  v0  +  2p  Y,  (1  -  ^)k~i~1  TTnj  .  (7) 
i= o 

The  first  term  is  clearly  a  transient  which  for  0  <  p  <  1 
will  vanish  after  a  sufficiently  large  number  of  signal  oc¬ 
currences.  Therefore,  at  steady-state  only  the  second  term 
in  (7)  will  be  important.  Taking  lini;>^ot  and  the  expected 
value  we  obtain 

lim  E  {vfc  }  =  -E{voo}  =  0,  (8) 

k—>o o 

since  the  noise  n j  is  assumed  to  be  zero-mean.  Accord¬ 
ingly  the  steady-state  weight  vector  is  an  unbiased  es¬ 
timator  of  the  Wiener  solution  (3). 

The  steady-state  weight  error  vector  variance  will  be 

^  ^ ^ 

££(l-2/x)fc_,_1 

3= 0  1=0 

TTE{nmj}T  (1  — .  (9) 

If  the  noise  signal  is  assumed  to  be  stationary  with  corre¬ 
lation  function  shorter  than  the  gap  between  consecutive 
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occurrences1,  then 

E  {miij}  =  5tj  N  (10) 

being  N  the  NxN  noise  covariance  matrix.  Accordingly, 
the  steady-state  weight  error  vector  energy  is 

k—  1 

£{v»v°o}  =  lim  V  V(l-2^)2(^1)fr{TTNT} 

k—>  oo 

j= 0 

=  — —  fr{TTNT|.  (11) 

1  -  p  1  J 

The  MSE  is  usually  decomposed  as  jBLMS  =  J°  +  J|x 
where  Jj?  is  the  MSE  at  the  optimum  and  the  excess  MSE 
J|x  can  be  written  as 

JF  =  E{vlvk}-2E{vlTTe°k}  (12) 
At  the  optimal  solution,  the  orthogonality  principle  applies 
TTeJ  =  Tt  (i  -  TTt  j  s  +  TTnk  =  Trnt .  (13) 

Hence  the  cost  functions  can  be  written  as 

jBLMS  =  jo  +  E  {vj[vfc}  -  2£{v[TTnt}  (14) 

Using  the  same  noise  assumption  as  in  (10)  the  cross 
term  E  {v;1T,n/,. }  at  steady-state  will  be  null  because 

fc-i 

lim  ^2(l-2n)k~j~1tr{TTE{nknj}T}=0.  (15) 
j=o 

Summing  up,  the  steady-state  MSE  will  be 

jBLMS  =  jo°  +  tj,  {TTNX  j  _  (16) 

1  p 

In  the  case  of  complete  expansions  we  have  that 
tr  {TtNT}  =  tr  {N}  which  is  equal  to  the  noise  energy. 
In  the  case  of  white  noise  and  incomplete  expansions 

•CMS  =  ■  (17) 

i  n 

It  may  be  worthwhile  to  point  out  certain  relationships 
to  the  LMS  algorithm.  In  the  case  of  the  LMS  algo¬ 
rithm,  J|x  is  composed  of  three  terms  [2,  Eq.  (12-13)] 
while  for  the  BLMS  algorithm  only  two  terms  are  present 
in  (14)  because  the  truncation  signal  error  is  orthogonal 
to  the  input  basis  functions  T.  Moreover,  the  LMS  algo¬ 
rithm  converges  to  a  biased  estimate  for  truncated  expan¬ 
sions  [2],  while  the  BLMS  estimation  is  steady-state  unbi¬ 
ased.  When  complete  expansions  p—  N  are  used  it  can  be 
noted  that  J^LMS  =  J^MS  [2].  This  result  agrees  with  the 
fact  that  the  algorithms  become  identical  when  complete 
expansions  are  used  (compare  (6)  with  [2,  Eq.  (8)]). 

1  The  very-low  frequency  components  of  biomedical  signals,  e.g.  base- 
line  wander  in  the  ECG,  are  usually  removed  in  a  pre-processing  stage 
because  they  do  not  convey  any  valuable  clinical  information. 


IV.  Equivalent  Transfer  Function 

Truncated  orthogonal  expansions  can  be  understood  as 
linear  time-variant  filters.  The  equivalent  instantaneous 
impulse  and  frequency  responses  were  calculated  in  [5] 
where  the  linear  coefficients  were  estimated  using  the  IP 
and  the  LMS  algorithm.  This  section  will  extend  the  anal¬ 
ysis  for  the  BLMS  algorithm. 

In  the  update  equations  of  BLMS  (6)  the  term  T7  d;. 
represents  the  IP  estimation  of  w°  using  only  information 
from  the  Ar-th  occurrence.  The  first  term  accounts  for  the 
estimation  done  at  the  previous  occurrence.  Consequently, 
the  BLMS  algorithm  can  be  understood  as  a  transform  do¬ 
main  exponential  averager.  It  is  well-known  that  exponen¬ 
tial  averaging  is  equivalent  to  a  linear  time-invariant  fil¬ 
ter  whose  transfer  function  is  a  comb  filter.  On  the  other 
hand,  truncated  orthogonal  expansions  estimated  with  in¬ 
ner  product  are  equivalent  to  a  linear  time-variant  filter  [5] . 
Therefore  the  combination  of  both  systems  is  a  linear  time- 
variant  filter. 

An  alternative  demonstration  can  be  done  by  looking 
at  the  reconstructed  signal.  A  first-order  finite  difference 
equation  is  obtained  by  premultiplying  both  sides  of  (6)  by 

t 

yfc+i  =  (1  -  2p)  yk  +  2pTTrdk  .  (18) 

In  the  case  of  complete  expansions  TTt  =  Ly ,  and  the 
coefficients  in  (18)  are  scalar  and  time-invariant.  When 
truncated  orthogonal  expansions  p  <  N  are  considered,  a 
coupled  system  of  finite  difference  equations  is  obtained 
from  (18)  because  TTt  ^  Ijv  . 


V.  Results 

The  performance  of  the  three  estimation  methods  (IP, 
LMS  and  BLMS)  is  illustrated  by  a  simulation  example 
in  which  the  characteristics  of  an  ECG  signal  is  studied. 
In  particular,  the  ECG  is  analyzed  with  respect  to  the  ST- 
T  complex  (Fig.  2)  since  this  part  of  the  cardiac  cycle 
frequently  reflects  myocardial  ischemia  (this  condition  is 
caused  by  a  lack  of  blood  supply  in  a  certain  region  of  the 
heart  wall).  Changes  that  occur  in  the  ST-T  complex  due 
to  ischemia  are  traditionally  quantified  by  the  amplitude 
measurement  “ST60”  obtained  60  milliseconds  after  the 
depolarisation  phase  has  ended. 

Basis  functions  derived  by  using  the  KL  expansion  have 
been  found  useful  for  monitoring  of  ischemia  [6].  The  KL 
basis  functions  used  in  the  present  study  were  estimated 
from  a  training  set  of  signals  including  several  databases  in 
order  to  adapt  the  basis  functions  to  a  large  variety  of  ECG 
morphologies.  The  four  most  significant  basis  functions 
are  also  plotted  in  Fig.  2.  It  should  be  emphasized  that 
although  the  KL  basis  functions  have  been  selected  here, 
other  orthogonal  expansion  can  be  used  as  well. 

The  signal  analyzed  below  was  synthesized  as  a  se¬ 
quence  of  identical  ST-T  complexes,  in  the  same  way  as 
was  done  in  [2],  to  which  Gaussian  white  noise  was  added 
with  an  SNR=  20  dB.  The  three  estimation  methods  (IP, 


Fig.  2.  ST-T  complex  selected  for  the  simulation  from  a  normal  heart¬ 
beat. 


Occurrence  number  Occurrence  number 


(a)  Bias  of  the  first  coef.  w*.  [1]  (b)  Variance  E  {vfev4 

Fig.  3.  Bias  and  variance  for  small  memory  (/z=0.3)  and  large  truncation 
error  (p  —  1)  which  corresponds  to  29.5%  of  the  signal  energy. 


LMS  and  BLMS)  were  then  applied  to  the  simulated  sig¬ 
nals.  Average  results  from  a  set  of  5000  trials  are  shown  in 
Figs.  3-5,  with  several  values  of  the  number  of  basis  func¬ 
tions  p  and  the  step-size  //.  The  results  below  presents  the 
performance  during  “steady-state”  heart  conditions,  how¬ 
ever,  it  is  naturally  of  interest  to  also  study  the  performance 
during  changes  in  the  ST-T  segment;  such  study  is  outside 
the  scope  of  the  present  paper. 

The  first  component  weight  error  vector  trajectory  is  il¬ 
lustrated  in  Fig.  3(a)  when  only  one  basis  function  is  used 
in  the  expansion  model  with  a  large  step-size  (/z  =  0.3). 
The  steady-state  bias  of  the  LMS  algorithm  is  large  due  to 
the  truncation  signal  error  and  the  large  value  of  the  step- 
size.  In  contrast,  BLMS  obtains  an  steady-state  unbiased 
estimate;  IP  is  unbiased  at  any  occurrence.  The  variance 
of  the  LMS  algorithm  is  shown  at  every  time  instant.  The 
large  variance  is  due  to  the  combination  of  large  truncation 
error  and  large  step-size. 

If  more  memory  is  used  by  the  adaptive  algorithms 
(lower  value  of  //  ),  the  steady-state  variance  will  be  lower, 
but  the  convergence  speed  will  decrease,  as  it  is  illustrated 
in  Fig.  4.  It  can  be  checked  that  the  LMS  and  BLMS  per¬ 
formance  are  very  similar  when  very  small  value  of  the 
step-size  are  used,  but  still  some  differences  due  to  the 
truncation  error:  the  LMS  is  biased  and  with  a  slightly 
higher  variance  at  steady-state. 

When  a  larger  number  of  basis  functions  is  used  in  the 
expansion,  most  of  the  signal  energy  is  contained  in  the 
signal  subspace  spanned  by  T,  and  the  effect  of  the  trunca¬ 
tion  error  on  the  LMS  is  much  less  important  (see  Fig.  5), 
even  for  large  values  of  //  (note  that  when  complete  ex¬ 
pansions  are  used  LMS  and  BLMS  are  equivalent  for  any 
step-size).  It  is  also  illustrated  in  Figs.  3  and  5  that  the 
number  of  basis  functions  used  in  the  expansion  has  a  crit- 


